1. Field of the Invention
The invention pertains generally to data streaming applications, and more particularly to methods and apparatus for estimating end-to-end distortion within a network which can allow optimizing streaming delivery strategies for pre-compressed data, such as video and other forms of media, and for utilizing distortion estimation within a rate-distortion framework for providing optimized resilient delivery of pre-compressed data streams.
2. Description of Related Art
The Internet has experienced explosive growth in recent years, and the once textual nature of network transmissions is shifting toward an increasing amount of streaming data (media) transfers, such as video streaming. As sufficient bandwidth and computational resources become available, video streaming applications have begun to attract increasing levels of attention. However, the heterogeneous and time-variant nature of current IP networks still presents a number of challenges for video coding algorithms and adaptive delivery schemes. One major requirement is to provide a robust video streaming system so that the perceived quality of the video stream varies in a graceful manner in response to periodic network quality of service (QoS) fluctuations.
It is important to distinguish between two main transmission paradigms for video communication over the networks, namely, live video and pre-compressed video. One major difference being that network status information is available during compression at the time of transmitting live video. When distributing live content, the encoder compresses and/or processes the raw data and passes the bit-stream to the network at the time of transmission. If the encoder has knowledge of the current status of the underlying network resources, it can as a result analyze the end-to-end behavior of the system in response to the video feed. Consequently, the streaming of live video allows efficient and flexible source and channel coding methodologies to be employed to optimize the performance of a given network for the video stream.
The above approach, however, is incompatible with the increasing number of applications that stream pre-compressed video over the network. It will be appreciated that for pre-compressed video the network conditions are not known at the time of compressing the video. Although numerous applications exist for streaming pre-compressed video, one area of widespread activity relates to streaming pre-compressed video delivered as “video on demand”. The raw video content for “video on demand” is compressed offline and stored on servers for later distribution over the network. The delivery of the streaming video is subject to whatever network conditions exist when the content is delivered according to customer demand. It should be noted that network conditions may vary in response to a number of parameters of the network, such as available bandwidth, packet loss probability, delay jitter, routing, availability of links between the server (transmitter) and client (receiver).
Variance of network conditions can substantially impact system performance, wherein employing adaptive source/channel coding techniques at the time of delivery can reduce variations in the perceived quality of received content. The optimization of adaptive strategies requires that the distortion of the reconstructed video at the receiver be estimated. The end-to-end distortion value (dB) quantifying the difference between the original raw media data and the decoder reconstructed signal taking into account compression, packet loss, and error concealment. It should be appreciated that the estimation of end-to-end distortion is fundamental to performing optimal transmission of pre-compressed video regardless of the application. However, a number of difficulties arise when estimating end-to-end distortion for a pre-compressed video stream for which parameters of the original video, prior to compression, are not available.
Attempts have been proposed for solving the problems with the streaming of pre-compressed video. It has been recognized that the ideal resilience strategy at the server is one which adapts to the actual bandwidth and packet loss statistics of the network in order to minimize the expected end-to-end distortion (i.e. the perceived distortion of the reconstructed video at the receiver). A Lagrangian rate-distortion (RD) framework was proposed to achieve the optimal adaptation strategy. The practical utility of the approach, however, is limited by the accuracy and efficiency of estimating end-to-end distortion.
It should be appreciated that the task of computing end-to-end distortion is complicated by a number of inter-related factors, including (prior) quantization, packet loss statistics, error resilience procedures, and error concealment. In addition, the use of inter-frame prediction in video coders causes spatial and temporal error propagation, and hence additional inter-dependencies between packets. Furthermore, unlike the coding and transmission of a live video stream, two important pieces of information are missing for a system delivering pre-compressed media. Specifically, the actual network status is unknown at the time of compressing the video stream, and the error resilience procedures employed at the time of delivery have no access to the original video. Therefore, the optimal error resilience approaches which are utilized for delivering live video are not suitable for use with pre-compressed video streams because the effective packet loss rate and original video data stream are not simultaneously available.
In order to render distortion estimation tractable, current approaches either neglect inter-frame error propagation, or ignore the effects of error concealment. The inaccuracies that arise from these limited approaches, however, can seriously compromise the performance of the adaptive strategies.
Therefore, a need exists for end-to-end distortion estimation methods that can readily determine expected distortion for pre-compressed video streams without ignoring inter-frame propagation and error concealment. Additionally, methods are needed for employing distortion estimations within an RD-framework for optimizing delivery of pre-compressed video streams. The present invention satisfies those needs, as well as others, and overcomes the deficiencies of previously developed distortion estimation techniques and adaptive transport tools.